Media Source Validation and Embedded Identification

ABSTRACT

Ways to associate source information with a media item are described. Source information may be validated. A media sourcing engine may receive a selected media item, collect source information associated with the selected media item, and associate the source information with the selected media item. The media source engine may be associated with, or otherwise utilized by, various publication platforms, such as social media sites, content hosting sites, etc. Source information may be associated with a media item by generating augmented media content that may be added to, embedded into, and/or otherwise associated with content associated with a received media item.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/341,013, filed on Jun. 7, 2021.

BACKGROUND

Various jurisdictions (e.g., national, state, local, etc.) may enforcevarious regulations regarding distribution of certain types of mediacontent (e.g., political advertising).

Therefore, there exists a need for a way to validate source informationand embed the validated source information into a media content item.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

The novel features of the disclosure are set forth in the appendedclaims. However, for purpose of explanation, several embodiments areillustrated in the following drawings.

FIG. 1 illustrates an example overview of one or more embodimentsdescribed herein, in which a media sourcing engine validates sourceinformation of a media item and associates the validated informationwith the media item;

FIG. 2 illustrates an example overview of one or more embodimentsdescribed herein, in which media source information is associated with amedia item by embedding a media clip;

FIG. 3 illustrates an example overview of one or more embodimentsdescribed herein, in which media source information is associated with amedia item by overlaying the information onto a media clip;

FIG. 4 illustrates an example overview of one or more embodimentsdescribed herein, in which media source information is associated with amedia item by overlaying the information onto an image;

FIG. 5 illustrates a schematic block diagram of an environmentassociated with one or more embodiments described herein;

FIG. 6 illustrates a data structure diagram of various data structuresassociated with one or more embodiments described herein;

FIG. 7 illustrates a flow chart of an exemplary process that associatessource information with a media item;

FIG. 8 illustrates a flow chart of an exemplary process that extractsand analyzes data associated with a media item;

FIG. 9 illustrates a flow chart of an exemplary process that identifiesrules associated with publication of a media item;

FIG. 10 illustrates a flow chart of an exemplary process that validatessource information for a media item;

FIG. 11 illustrates a flow chart of an exemplary process that generatesan augmented media item including source information;

FIG. 12 illustrates a flow chart of an exemplary process that appliesmachine learning to models associated with media source informationvalidation and association; and

FIG. 13 illustrates a schematic block diagram of one or more exemplarydevices used to implement various embodiments.

DETAILED DESCRIPTION

The following detailed description describes currently contemplatedmodes of carrying out exemplary embodiments. The description is not tobe taken in a limiting sense, but is made merely for the purpose ofillustrating the general principles of some embodiments, as the scope ofthe disclosure is best defined by the appended claims.

Various features are described below that can each be used independentlyof one another or in combination with other features. Broadly, someembodiments generally provide ways to validate media item sourceinformation and associate the validated source information with themedia item. A media sourcing engine of some embodiments may receive aselected media item, validate source information associated with theselected media item, and associate the validated source information withthe selected media item. The media source engine may be associated with,or otherwise utilized by, various publication platforms, such as socialmedia sites, content hosting sites, etc.

Validated source information may be associated with a media item bygenerating augmented media content that may be added to, embedded into,and/or otherwise associated with content associated with a receivedmedia item.

Although many examples herein may refer to “source” or “sourcing”information, “validation” information, and/or other specified examplesof augmented information or content, one of ordinary skill in the artwill recognize that various other types of information and/or contentmay be associated with an augmented media item by generating augmentedmedia content and/or other information to include with the augmentedmedia item. For instance, upload date or publication date informationmay be associated with the augmented media item.

FIG. 1 illustrates an example overview of one or more embodimentsdescribed herein, in which a media sourcing engine 100 validates sourceinformation of a media item 110 and associates the validated informationwith an augmented media item 120. As shown, a request for sourcevalidation and media item processing may be received by the mediasourcing engine 100 from a publishing platform 130. Such a request mayinclude the media item 110 and/or identifying information associatedwith the media item (e.g., a file name or other identifier, address suchas a uniform resource locator (URL), etc.) and/or other relevantinformation such as identifying information related to the publishingplatform 130, associated user(s), etc.

The media item 110 may include various types of media content (e.g.,text, graphics, photographs, video, audio, etc.) and may be associatedwith various different file types, numbers of files, and/or otherappropriate storage attributes. The media item 110 (and/or associatedrequest for publication) may be received from various appropriatesources across various appropriate channels. For instance, the mediaitem 110 may be received from a resource such as a user device and/orother user equipment (e.g., a smartphone, tablet, laptop, personalcomputer, wearable device, etc.) via a network-based communicationchannel. As another example, the media item 110 may be received from aresource such as a content server via an application programminginterface (API).

In some embodiments, publishing platform 130 (and/or other appropriateresources) may provide a set of approved media items for selection byusers. For instance, a set of approved political advertisements may beprovided for selection by users of a social media site.

In this example, media item 110 may be an audiovisual media file such asan advertisement. A social media site may serve as the publishingplatform 130. A user of the site may submit a request to publish themedia item 110 and/or otherwise initiate publication of the media item110 (e.g., by uploading the media item 110 to a resource such as apublic forum, group page, personal page, etc.).

Media item 110 may include a header 140, media content 145, and/or othercomponents. The header 140 may include information and/or metadata suchas file name, type, size, author, title, etc. In some embodiments, theheader 140 may include information related to content attributes, such aresolution, video speed, audio type, start time, stop time, scene orother markers or delimiters, etc. The media content 145 may include dataassociated with various types of media content, such as photographs,graphic content, text, video, audio, and/or combinations of such mediacontent (e.g., data associated a video that includes audio and overlaidgraphics and text).

Publishing platform 130 may be a network-accessible resource associatedwith provision of multimedia content. Examples of such platforms includesocial media sites and/or other web sites, communication or messagingplatforms, email services, internet service providers (ISPs), privatenetworks (e.g., a corporate intranet), streaming services, etc. Thepublishing platform 130 may be implemented using various systems ordevices, such as servers, storages, network interfaces, etc.

In this example, publishing platform 130 may be a social media site, andmedia item 110 may be an audiovisual clip showing a politicaladvertisement that is uploaded (or otherwise received) from a user ofthe social media site.

Media sourcing engine 100 may be implemented using various devicesand/or components, such as servers, storages, network interfaces, etc.In some embodiments, the media sourcing engine 100 may be at leastpartly implemented by, at, and/or via the publishing platform 130 (e.g.,as an application or service running on the publishing platform 130). Inthis example, media sourcing engine 100 includes a media analyzer 150,validation engine 160, one or more media editors 170, and a rulesrepository 180.

The media sourcing engine 100 may receive the request from a publishingplatform 130 or other appropriate resource. The augmented media item 120may be returned to the publishing platform 130 and/or provided to otherresources, such as a different requestor, a media server, etc. Theaugmented media item 120 and/or identifying information associated withthe media item (e.g., a file name or other identifier, address such as aURL, etc.) and/or other relevant information such as identifyinginformation related to the publishing platform 130, associated user(s),etc. may be provided to the publishing platform 130 and/or otherappropriate resource via a notification or response message and/or viaother appropriate ways (e.g., by adding the augmented media item 120 toa storage or server accessible via API).

Media analyzer 150 may analyze and/or process the received media item110. The media analyzer 150 may communicate with other components ofmedia sourcing engine 100 and/or at least partly control or direct suchother components. The media analyzer 150 may analyze the received mediaitem 110 to determine relevant attributes associated with the mediaitem, such as media type, file type, video resolution, video speed,audio type, audio resolution, start time, stop time, scene or othermarkers or delimiters, etc. Media analyzer 150 may analyze media content145, header information 140, and/or other information associated withmedia item 110 (e.g., publisher, file name, etc.).

The media analyzer 150 may send a request for source validation to thevalidation engine 160. The request for source validation received frompublishing platform 130 may include information such as username,platform, etc. In addition, identifying information such as internetprotocol (IP) address of the requestor may be received or determined.The validation engine 160 may receive the request for source validationfrom the media analyzer 150 and, based on analysis of the request,identify and apply rules stored at rules repository 180.

The rules repository 180 may include various validation rules and/orother rules associated with validation and/or publication of content.For instance, a particular publishing platform 130 such as a socialmedia site may have a set of rules associated with user verification orvalidation (e.g., by requiring a username and/or password to matchstored values, by evaluating a key or token provided via publishingplatform 130, etc.). As another example, various sets of rules may beassociated with various jurisdictions, regulatory agencies, and/or otherparties that may enforce rules associated with publication of content(e.g., campaign finance laws requiring indication of sourcinginformation, disclosure of funding sources, association of a candidateto a political advertisement, etc.).

The rules repository 180 may apply various machine learning algorithmsto the various rules or models associated with the rules repository 180.Such machine learning may be based upon various types of feedback and/orother inputs. For instance, a human reviewer may indicate whethersourcing information appeared correct, whether the content was clear andcomprehensible, whether any rules were incorrectly enforced or applied,whether relevant sourcing requirements were met, etc.

The validation engine 160 and/or rules repository 180 may identifyrelevant rules. In some embodiments, the media analyzer 150 and/orvalidation engine 160 may analyze received content to match the contentto known or supported content items. For instance, a politicaladvertisement may be uploaded by a user and the content may be analyzedand matched to content associated with a site or resource that isassociated with a political action committee (PAC), publisher (e.g., anews or media company), and/or other content generator or provider.

The validation engine 160 may apply validation rules and provide avalidation response message indicating whether the sourcing has beenvalidated and/or meets any applicable requirements (e.g., if validationis not required the validation message may indicate that no validationwas necessary). The validation engine 160 may utilize various validationresources in addition to, or in place of, rules repository 180. Forinstance, username and/or password information may be sent to athird-party resource for validation or verification. As another example,if an email address is provided, a verification message may be sent tothe email address for further action. As still another example, matchingrequests may be sent to various content servers or other contentproviders. As yet another example, credit card information (e.g., cardnumber and billing address) may be collected and validated through apayment processing resource. In some embodiments, the validation engine160 and/or rules repository 180 may store, catalog, or otherwisemaintain a database of content elements that may be used for matching oranalysis.

In some embodiments, validated sourcing information may include a chainof sources related to a content item. For instance, source informationmay indicate that a political advertisement was generated by aparticular PAC and uploaded to a particular social media site by aparticular user.

Additional sourcing information may be appended, for example, if thecontent item is subsequently published or shared to another social mediasite by another user. The validation response message may include a setof rules to be enforced or applied by media analyzer 150 (e.g., rulesregarding content, sourcing, etc. that are applied by modifying thereceived media item 110).

The media analyzer 150 may receive the validation response message and,if the message indicates that the source is not able to be validated,may respond to the request from the publishing platform 130 with arejection message or other indication of failure to validate the sourceinformation.

If the validation response message indicates that the sourcing is valid(or that source validation is not necessary), the media analyzer 150 mayapply relevant received rules to processing of the received media item110. Continuing the political advertisement example, the set of rulesmay include a requirement that source information be provided via thepolitical advertisement. Rules may be associated with various mediaediting or generating operations. For instance, a rule may indicate thatsource information for an audiovisual file should be provided viaappended audiovisual content.

The media analyzer 150 may provide augmentation instructions to themedia editor(s) 170. The augmentation instructions may includeinformation related to media item 110, such as header 140 and/or mediacontent 145. The augmentation instructions may include sourceinformation (e.g., source name, publishing resource, etc.), sourcevalidation information (e.g., IP address, user device identifyinginformation, etc.), and/or associated sourcing rules.

Media editor(s) 170 may implement the received augmentation instructionsto generate the augmented media item 120. The augmented media item 120may include a modified header 190, media content 145, source indicationcontent 195, and/or other content or information (e.g., metadata, header140, etc.). The source indication content 195, and/or other augmentedcontent, may be merged with media content 145 in some embodiments.Source indication content 195 may generally be minimized relative to themedia content 145. For example, if an advertisement is thirty secondslong, the source indication content 195 may be limited to two seconds orfive seconds.

In some embodiments, modified header 190 and/or source indicationcontent 195 may include cryptographic data such as blockchain data. Suchcryptographic data may allow augmented media item 120 to serve as anon-fungible token (NFT). The cryptographic data may be generated invarious appropriate ways and embedded into the modified header 190 orotherwise be appropriately associated with augmented media item 120.

Continuing the above example, validated source information for thepolitical advertisement (e.g., publishing platform 130, username oridentifying information (e.g., personal information such as name,affiliations, etc.), publication time, original source (if known), etc.)may be received by media editors 170 and used to generate audiovisualcontent indicating the validated sourcing information. For instance,text-based sourcing information may be converted to audio content (e.g.,“text-to-speech”) that is appended to the end of augmented media item120. Such conversion may include modifying or augmenting the receivedsource information. For instance, if the name of an individual isreceived (e.g., “Bob Smith”), the name may be added to a standard textphrase (e.g., “content uploaded by”). As another example, graphicalsourcing information may be overlaid onto the media content 145. Asanother example, video content may be generated to match the audiocontent (e.g., an avatar or simulated entity such as a virtualnewsreader or celebrity).

The augmented media item 120 may be returned to the publishing platform130 and/or provided to other resources, such as a different requestor, amedia server, etc. The media sourcing engine 100 may provide associatedsourcing information and/or source validation information (e.g.,validation resource(s), rules applied, etc.) with the augmented mediaitem 120. In some embodiments, the augmented media item 120 may betagged or otherwise indicate that sourcing information has beenappended. Thus, for instance, a user may be able to submit a media item110 to media sourcing engine 100 and receive an augmented media item 120that is tagged or otherwise indicates that the augmented media item 120is ready for publication. Such tagging may be associated with a set ofpublishing resources, such as a set of social media sites or contenthosting resources, such that a user may generate one augmented mediaitem 120 that may then be distributed to the various resources.

One of ordinary skill in the art will recognize that although variousexamples in this disclosure may refer to specific types of content andanalysis (e.g., political advertisements and source validation),different embodiments may be associated with various different types ofcontent and/or analysis. For instance, some embodiments may analyzereceived content to determine if the content is suitable for viewing bychildren and automatically generate and insert a warning if the contentis not suitable.

FIG. 2 illustrates an example overview of one or more embodimentsdescribed herein, in which media source information is associated with amedia item 110 by embedding a media clip 210 that includes sourceindication content 195. In this example, the media clip 210 is added tothe end of media content 145. Different embodiments may include themedia clip 210 at various locations relative to media content 145 (e.g.,inserted before the content, inserted in the middle of the content,etc.). In this example, the media content 145 may be thirty secondslong, while the appended clip 210 is two seconds. Different embodimentsmay implement various rules or limitations related to size or length ofthe clip 210 relative to the size or length of the media content 145.The media clip 210 may include various media types, such as video,audio, graphics, etc.

FIG. 3 illustrates an example overview of one or more embodimentsdescribed herein, in which media source information is associated with amedia item 110 by overlaying media content 220 that includes sourceindication content 195 onto the media content 145. For instance,text-based graphics or subtitles may be generated and rendered onto anaudiovisual clip 110. The media content 220 may be added to the mediacontent at various locations, based on various relevant factors. Forexample, audio content 220 may be generated and overlaid onto a portionof the media content 145 that is determined to have no sound (or onlybackground music). As another example, text-based graphics may beinserted onto video content that is determined to have a homogeneousbackground.

FIG. 4 illustrates an example overview of one or more embodimentsdescribed herein, in which media source information is associated with amedia item 110 by overlaying the source indication information 230 ontoan image 145. In this example, media item 110 may include a single frameor image 145 and the source indication information 230 is overlaid ontothe bottom right corner of the image 145. The source indicationinformation 230 may be overlayed at a location determined to meetvarious criteria. For instance, the image 145 may be analyzed toidentify regions with solid or homogeneous backgrounds, colors may beanalyzed to determine locations with potential for high contrast, etc.

FIG. 5 illustrates a schematic block diagram of an environment 500associated with one or more embodiments described herein. As shown, theenvironment may include a media sourcing engine 100, a publishingresource 510, a media editor 520, a validation resource 530, a userdevice 540, a content server 550, and a rules repository 560.Environment 500 may be implemented using various devices and/or sets ofdevices and may be implemented using various networks or othercommunication pathways.

Publishing resource 510 may include various servers, storages, etc. thatmay be associated with one or more publishing platforms 130. Suchpublishing resources 510 may be able to communicate across, or beaccessible via, various network communication pathways. In someembodiments, publishing resource 510 may implement and/or includevarious other elements of environment 500, such as media sourcing engine100, media editor 520, validation resource 530, content server 550,and/or rules repository 560.

Media editor 520 may include various servers, APIs, etc., that mayprovide media editing capabilities. Media editor 520 may be accessibleacross one or more networks or other appropriate communication pathways.In some embodiments, media editor 520 may be implemented by, and/orincluded at, media sourcing engine 100.

Validation resource 530 may include various servers, APIs, storages,etc. that may provide source validation. Such resources may include, forinstance, social media resources, email account resources, and/orpayment processing resources (e.g., credit card processing services).

User device 540 may be any device or component that is able to interactwith the media sourcing engine 100 or the publishing resource 510.Examples of user devices 540 include smart phones, tablets, personalcomputers, laptops, wearable devices, embedded devices, etc.

Content server 550 may include various servers, APIs, storages, etc.that may provide or store media content. Such content servers 550 may beassociated with various resources, such as media hosting sites, socialmedia sites, publishing platforms 130, and/or other appropriateresources.

Rules repository 560 may include various servers, storages, etc. thatmay store and provide rules for use by the media sourcing engine 100and/or other system components, such as validation resources 530.

As described above, in some embodiments, media sourcing engine 100 mayimplement or include elements such as media editor 520, validationresource 530, content server 550, and/or rules repository 560.

One of ordinary skill in the art will recognize that environment 500 maybe implemented in various different ways without departing from thescope of the disclosure. For instance, various additional components maybe included, or some listed components may be omitted in someembodiments.

FIG. 6 illustrates a data structure diagram of one or more embodimentsdescribed herein, in which various profiles 610-640 are used to defineand select assessment elements and features. In this example, the set ofprofiles includes a platform profile 610, a rule profile 620, ajurisdiction profile 630, and a media profile 640. Different embodimentsmay include various different types of profiles associated with variousdifferent sets of elements. For instance, some embodiments may includeone or more media editor profiles associated with one or more mediaeditors 170 or 520. As another example, some embodiments may include oneor more source profiles associated with sources such as user accounts,social media groups, etc.

As shown, the platform profile 610 may include elements such as a uniqueidentifier, a listing of associated rules, a listing of associatedvalidation resources, a listing of associated content servers, a listingof associated platforms, and/or other appropriate elements. Suchlistings of associated elements may include, for instance, elementidentifiers, names, etc. that may be used to identify elementsassociated with a platform profile 610. Each platform profile 610 may beassociated with a resource such as publishing platform 130 and/orpublishing resource 510.

The rule profile 620 may include elements such as a unique identifier, arule type (e.g., validation rule, jurisdiction rule, etc.), a set ofmatching models (e.g., machine learning or artificial intelligencemodels) that may be used to identify elements (e.g., media attributes,platform attributes, user attributes, jurisdiction attributes, etc.)that match a given rule profile 620 and associated rule, and/or otherappropriate elements.

The jurisdiction profile 630 may include elements such as a uniqueidentifier, a listing of associated rules, a listing of validationresources, and/or other appropriate elements. Each jurisdiction profile630 may be associated with a publishing platform type (e.g., socialmedia site, media hosting site, etc.), geographic region, governingbody, and/or other appropriate entities.

The media profile 640 may include elements such as a unique identifier,a listing of associated media types, a listing of associated matchingmodels, a listing of associated rules, a listing of associated contentservers, and/or other appropriate elements.

Such profiles may allow various entities, media types, etc. to beassociated in various ways. For instance, a publishing platform 130 maybe owned by a corporation headquartered in a particular geographicregion. The associated platform profile 610 may indicate a set ofassociated rules, jurisdiction profiles, and validation resources thatare appropriate for and/or indicated by the governing body or otherentity associated with the particular geographic region.

As another example, a media profile 640 may be associated with adisputed claim or otherwise flagged content. Received media content thatmatches the profile may have various labelling rules applied (e.g., toindicate that a presented claim is disputed).

As another example, a platform profile 610 may include a listing ofmedia items that may be published by users of the platform. Such alisting may include, for instance, media items expressing approval ordisapproval with respect to a matter or topic. As another example, alisting may include media items associated with a set of qualifiedcandidates for public office (e.g., a listing including at least onemedia item for each candidate meeting the qualifications to be listed ona ballot).

FIG. 7 illustrates an example process 700 for associating sourceinformation with a media item. The process may generate an augmentedmedia item that includes content associated with the source information.Such source information may be verified or validated using variousvalidation resources. The process may be performed, for instance, when auser uploads or otherwise submits a media item for publication to aplatform. In some embodiments, process 700 may be performed by mediasourcing engine 100.

As shown, process 700 may include receiving (at 710) a media item or aselection of a media item, such as media item 110. The media item orselection may be received via a resource such as user device 540 and/orpublishing platform 510. In some embodiments, a resource such as mediaanalyzer 150 may receive the media item. If the received media itemincludes an address such as a URL, the process may retrieve the mediaitem from a storage or other resource (e.g., publishing platform 510 orcontent server 550).

Process 700 may include analyzing (at 720) the received media item. Suchanalysis may utilize various types of media item attributes or data,algorithms, etc. The analysis may be performed by a resource such asmedia analyzer 150. Analysis may include determining or identifyingattributes such as media type (e.g., audio, video, graphics, etc.),content type (e.g., advertising content, political content, etc.), mediasource, etc. Analysis may include comparing the media item to variousmodels in order to identify matching attributes (e.g., inclusion ofadult content, political or election-related content, etc.). In someembodiments, matching models may include specific entities (e.g., a listof celebrities, politicians, businesses, etc.) and/or content (e.g.,clips of public events, speeches, etc.) that may be associated withvarious profiles.

The process may include identifying (at 730) associated rules based onthe media item analysis and/or other relevant factors (e.g., media itemsource, publishing platform, etc.). Various types of rules may beassociated with various types of matching models, profiles, contenttypes, regions or jurisdictions, other rules, etc. Rules may beidentified by resources such as the validation engine 160, rulesrepository 180, and/or other appropriate resources.

As shown, process 700 may include receiving and validating (at 740)sourcing information based on the rules identified at 730. Suchvalidation may be performed using resources such as validation engine160, rules repository 180, validation resources 530, and/or otherappropriate resources. As an example, users who would like to publishcontent to a social media site may be required to submit credit cardinformation for verification via a payment processing resource (e.g., bycharging a nominal fee to publish content).

Process 700 may include generating (at 750) an augmented media item thatincludes the validated sourcing information. The validated sourceinformation may include different levels of disclosure, depending onassociated rules. For instance, in some cases the name of an individualuser may be validated and included in the augmented media item. Asanother example, a username or other semi-anonymous label may be used toindicate sourcing information relative to a particular platform (e.g., asocial media site). The augmented media item may include other contentthan sourcing information, such as required disclaimers associated witha particular jurisdiction. Depending on the type of media item received,various different types of content may be added to, embedded into,and/or otherwise combined with the received media item. For instance, anaudiovisual clip including the augmented content may be added to areceived audiovisual media item. As another example, a graphical overlay(e.g., a text overlay) with augmented content may be added to a stillimage. As yet another example, audio content may be added to an audio oraudiovisual media item.

The process may include providing (at 760) the augmented media item. Theaugmented media item, such as augmented media item 120 may be providedvia a file transfer, message indicating a file location or address,and/or other appropriate ways (e.g., by copying the augmented media itemto a content server 550 or other appropriate resource).

FIG. 8 illustrates an example process 800 for extracting and analyzingdata associated with a media item. The extracted and analyzed may beused for source validation, media item augmentation, rule identificationor selection, etc. The process may be performed when a media itempublication request is received. In some embodiments, operation 720described above may include process 800. In some embodiments, process800 may be performed by media sourcing engine 100 and/or a resource suchas media analyzer 150.

As shown, process 800 may include extracting (at 810) metadata from thereceived media item. Such metadata and/or other information may beextracted from a header, such as header 140, associated with a receivedmedia file structure. Metadata or file information may be extracted invarious appropriate ways depending on the file type, header type, etc.Such metadata may include information such as clip author, creationdate, title, etc.

Process 800 may include analyzing (at 820) the received metadatainformation. Such analysis may include identifying relevant attributes,matching profiles, etc. In addition to, or in place of, metadata,various other data associated with a media item may be analyzed, such asfile name, extension, creation date, modification date, etc.

The process may include extracting (at 830) content elements from thereceived media item. Such content extraction may include, for instance,extraction of video or audio data from an audiovisual file. As anotherexample, overlaid text or graphics may be extracted from an audiovisualfile. As still another example, scene or transition information may beextracted from a media item.

As shown, process 800 may include analyzing (at 840) the extractedcontent elements. Such analysis may include comparing the extractedcontent elements to various matching models. Different types of contentmay be analyzed in various different ways, as appropriate. For instance,audio content may be analyzed to identify speech and convert the speechto text for text-based analysis and matching. As another example, videocontent may be analyzed to identify strings of frames with consistentbackground or foreground graphics such that augmented content may beoverlaid in areas that will promote visibility. As another example,audio content may be converted to text and the text may be compared tovarious matching terms, phrases, etc. Such an approach may be used toidentify inappropriate or adult-themed content, to flag content withoffensive language, etc.

Process 800 may include providing (at 850) the extracted data andanalysis to other resources. For instance, data extracted by mediaanalyzer 150 may be provided to resources such as validation engine 160in order to identify relevant rules, or to resources such as mediaeditors 170 for use in generating augmented content.

FIG. 9 illustrates an example process 900 for identifying rulesassociated with publication of a media item. The rules may be identifiedusing resources such as profiles 610-640. The rules may specify sourcingand/or other requirements associated with publication of content. Theprocess may be performed when received media has been analyzed orpartially analyzed. In some embodiments, operating 730 described abovemay include process 900. In some embodiments, process 900 may beperformed by media sourcing engine 100.

As shown, process 900 may include receiving (at 910) extracted data andanalysis. Such data may be received from a resource such as mediaanalyzer 150 and may include metadata, content data, extractedinformation, matching models, etc. Data may be extracted using a processsuch as process 800.

Process 900 may include receiving (at 920) platform information.Platform information may be received from a resource such as publishingplatform 130, publishing resource 510, and/or other appropriateresources. Such information may include information such as anidentifier of a platform profile 610. The platform profile 610 mayinclude associated rules, matching models, other profiles, etc.

The process may include receiving (at 930) source information. Sourceinformation may be received from a resource such as publishing platform130, publishing resource 510, user device 540, and/or other appropriateresources. Source information may include information related to a useraccount, personal information such as name, financial information suchas credit card information, and/or other relevant source information. Insome cases, source information may include a creator or author of theassociated content, copyright owner or other such entity, a chain ofpublication sources, and/or other such information related to thepublication history of a media item. Source information may includeinformation related to an entity or person initiating a publication orrepublication of a media item.

As shown, process 900 may include identifying (at 940) matching rulesand/or models. Such rules or models may be identified in variousappropriate ways. For instance, a platform profile 610 associated withthe publishing platform may include a listing of associated rules. Asanother example, a jurisdiction profile 630 associated with a geographicregion may include a listing of applicable associated rules. As stillanother example, a media profile 640 associated with the received mediatype may include a listing of associated rules. Matching rules and/ormodels may be associated with other matching rules or models (e.g., byincluding a list of identifiers associated with such matching rules ormodels in a given matching rule or model). A resource such as a listingor table of matching rules (e.g., a listing of rule profile 620identifiers) may be generated.

FIG. 10 illustrates an example process 1000 for validating sourceinformation for a media item. Such a process may validate or verify theidentity of a source for a media item that is submitted or selected forpublication. The process may be performed when a media item is submittedfor publication. In some embodiments, operation 740 described above mayinclude process 1000. In some embodiments, process 1000 may be performedby media sourcing engine 100, and, specifically, by a resource such asvalidation engine 160.

As shown, process 1000 may include receiving (at 1010) sourceinformation. A “source” may be any entity that provides a media item forpublication. For example, a source may be a user of a publishingresource, a content creator, an organization or business entity, etc. Insome embodiments, a source may be an electronic device such as a userdevice or server. Source information may be received from a resourcesuch as publishing platform 130, publishing resource 510, user device540, and/or other appropriate resources. Source information may bepassed from a resource such as media analyzer 150 to a resource such asvalidation engine 160. Source information may include identity oridentifying information (e.g., name, username, organization name, etc.),connection or device information (e.g., IP address, device serialnumber, etc.), and/or other appropriate information that may identify ordefine a source. Source information may be associated with, and/orinclude, validation or confirmation information (e.g., a credit cardnumber, address, email address, a state-issued identification numbersuch as a driver's license number, and/or other types ofidentity-validating information).

Process 1000 may include receiving (at 1020) validation rules.Validation rules may include various elements and/or criteria associatedwith source validation and/or other types of verification or validation.For instance, a rule associated with a social media site may requirevalidation of user login information associated with the social mediasite. As another example, an upload to a media hosting site may requirevalidation of a responsible individual through financial informationsuch as a credit card.

The process may include identifying (at 1030) validation resources. Insome embodiments, the received validation rules may include referencesto various associated validation resources. In some cases, a resourcesuch as validation engine 160 may identify or select various resourcesbased on a type of validation and/or other appropriate rule attributes.For instance, a set of credit card processing resources may be used tovalidate financial information, where the set of credit card processingresources may be indicated by a validation rule, a set of defaultresources associated with the media sourcing engine 100, a publishingplatform profile 610, etc. As another example, validation resources maybe identified based on geographical location of a source or publishingresource.

As shown, process 1000 may include extracting (at 1040) validationinformation. Validation information may be received at the validationengine 160 from a resource such as media analyzer 150. Validationinformation may be extracted or identified based on various associatedrules, profiles, available validation resources, etc. For instance, if avalidation rule indicates that a source's email must be verified, emailinformation may be extracted from information received from thepublishing platform 130. Continuing the example, if email information isnot available or not able to be identified, the validation engine 160and/or other appropriate resources may send a request for theinformation (e.g., by sending a request for source email address to thepublishing platform 130).

Process 1000 may include applying (at 1050) validation rules.Application of validation rules may include, for instance, comparison ofvalidation data to various evaluation criteria. For instance, a usernameand/or password may be matched to a database entry associated with asocial media site user. As another example, financial and identifyinginformation (e.g., credit card information, name and address ofcardholder, etc.) may be submitted to a payment processing resource. Asanother example, a supplied email and/or name may be compared to variousdatabases or other resources. As still another example, resources suchas search engines may be used to validate public information. Validationrules may include various criteria for approval or denial of suchvalidation based on the appropriate validation data. For instance, aresponse to a request for validation of username or password may includean indication of success or failure.

The process may include providing (at 1060) validation results.Validation results may be provided by a resource such as validationengine 160 to a resource such as media analyzer 150. Validation resultsmay include, for instance, a discrete result indication (e.g.,“success”, “failure”, etc.), a listing of validated information (e.g.,name, username, email, etc.), a listing of validated information toinclude in augmented content (e.g., name of source, publishing platform,etc.), a listing of validation resources used to validate theinformation, a listing of rules and/or profiles used to perform theevaluation, and/or other relevant information.

In some cases, validation results may include information received froma resource such as a source or publishing platform 130, where theinformation is not examined or validated with respect to any otherresource. For instance, if a request is received from a publishingplatform 130 that indicates only the name of the publishing platform forsource information, the received information may be included in anaugmented media item without any verification or validation other thanthe indication provided by the requestor itself.

FIG. 11 illustrates an example process 1100 for generating an augmentedmedia item including source information. Such a process may be used togenerate augmented content based on validated information and includethe augmented content with content in a received media item, such asmedia item 110. The process may be performed after validationinformation is generated. In some embodiments, operation 750 describedabove may include process 1100. In some embodiments, process 1100 may beperformed by media sourcing engine 100, and, specifically, by mediaanalyzer 150 and/or media editors 170.

As shown, process 1100 may include receiving (at 1110) a media item. Themedia item, such as media item 110, may be received at a resource suchas media editor 170. In some embodiments, various augmentationinstructions may be received. Such augmentation instructions mayinclude, for instance, augmented content type (e.g., audio, audiovisual,text overlay, etc.), augmented content data (e.g., validated sourceinformation, required disclaimers, etc.), and/or other appropriateinformation or instructions.

Process 1100 may include extracting (at 1120) metadata. Metadata mayinclude any data associated with the received media item that is notconsidered media content. For example, metadata may include headerinformation, such as header 140, file path information, file attributes,etc. Metadata may be extracted by a resource such as media analyzer 150.

The process may include identifying (at 1130) media types. Media typeinformation may be identified or determined by a resource such as mediaanalyzer 150. Media type information may include, for instance, filetype (e.g., mp3, way, mov, etc.), content type (e.g., audio,audiovisual, graphic, etc.), streaming or transmission protocolinformation, and/or other relevant information.

As shown, process 1100 may include extracting (at 1140) content. Contentmay be extracted in various appropriate ways, depending on variousrelevant factors, such as content type, augmentation type, validationdata, etc. Content may be extracted by resources such as media analyzer150 and/or media editor 170. For example, if a received media item, suchas media item 110, includes audiovisual content, audio content data andvideo content data may be extracted. As another example, if augmentedcontent includes only an audio disclaimer, audio data may be extracted.

Process 1100 may include receiving (at 1150) sourcing information. Suchsourcing information may include validated and/or unvalidatedinformation associated with a source of the received media item.Sourcing information may be provided in various appropriate formats(e.g., text, references or pointers, etc.).

The process may include receiving (at 1160) validation information.Validation information may include information such as validation type,a listing of validation resources, validation result, etc. Validationinformation may be received from a resource such as validation engine160.

As shown, process 1100 may include generating (at 1170) augmented mediacontent. Such augmented media content may include information such assource information, validation information, etc. Some embodiments maygenerate text-based augmented data (e.g., required disclaimers, sourceinformation, etc.) that may be converted into various formats, asappropriate. For instance, text may be converted to speech to generatedaugmented audio content. As another example, a text overlay or set ofsubtitles may be generated based on the text. In some embodiments,augmented media content may be generated using resources such asthree-dimensional modeling and/or animation. For instance, audio contentmay be “presented” by an animated or rendered video avatar.

Process 1100 may include updating (at 1180) metadata. Depending on thefile type and/or other relevant factors, updated file information may beadded or associated in various appropriate ways. For instance, a header,such as header 140 may be updated to include additional and/or differentinformation in a modified header 190. Updated metadata information mayinclude, for instance, information related to source, validation,publisher, etc. In some embodiments, information such as previoussourcing or validation information may be extracted from metadataassociated with a received media item and appended to current sourcingor validation information such that a sourcing “chain” is generated ormaintained. In some embodiments, such information may include blockchainor other such crypto technology that may allow for the sourceinformation and/or other information to be embedded into or otherwiseassociated with an augmented media item, such as augmented media item120.

The process may include generating (at 1190) an augmented media item.The augmented media item, such as augmented media item 120, may begenerated in various appropriate ways, depending on the media itemcontent type(s), augmented content type(s), and/or other relevantfactors. Resources such as media editor 170 may generate and addaugmented content to received content and/or modify received content toinclude augmented content.

FIG. 12 illustrates an example process 1200 for applying machinelearning to models associated with media source information validationand association. Such a process may be used to update models as feedbackbecomes available and/or additional content is processed. The processmay be performed at regular intervals and/or other appropriate times. Insome embodiments, process 1200 may be performed by media sourcing engine100.

As shown, process 1200 may include receiving (at 1210) feedback.Feedback may be received in various ways from various appropriatesources. For example, users of a social media site may indicate whetherappended information, such as sourcing information, was accurate and/orrelevant. As another example, an administrator or regulator may reviewpublished content and indicate whether applicable publishing orreporting requirements were satisfied.

Process 1200 may include receiving (at 1220) available machine learningmodels. Based on the received feedback, relevant models may beidentified and retrieved.

The process may include updating (at 1230) models and/or generating newmodels based on the received feedback. For instance, if a matching modelis associated with positive feedback, the matching model may be weightedmore heavily in analysis or made more likely to be associated with aprofile. In contrast, if a matching model is associated with negativefeedback, the matching model may be weighted less heavily in analysis ormade less likely to be associated with a profile.

As shown, process 1200 may include storing (at 1240) the updated models.The updated or newly generated models may be stored for use in futureanalysis and machine learning.

One of ordinary skill in the art will recognize that processes 700-1200may be implemented in various different ways without departing from thescope of the disclosure. For instance, the elements may be implementedin a different order than shown. As another example, some embodimentsmay include additional elements or omit various listed elements.Elements or sets of elements may be performed iteratively and/or basedon satisfaction of some performance criteria. Non-dependent elements maybe performed in parallel.

The processes and modules described above may be at least partiallyimplemented as software processes that may be specified as one or moresets of instructions recorded on a non-transitory storage medium. Theseinstructions may be executed by one or more computational element(s)(e.g., microprocessors, microcontrollers, digital signal processors(DSPs), application-specific integrated circuits (ASICs), fieldprogrammable gate arrays (FPGAs), other processors, etc.) that may beincluded in various appropriate devices in order to perform actionsspecified by the instructions.

As used herein, the terms “computer-readable medium” and “non-transitorystorage medium” are entirely restricted to tangible, physical objectsthat store information in a form that is readable by electronic devices.

FIG. 13 illustrates a schematic block diagram of an exemplary device (orsystem or devices) 1300 used to implement some embodiments. For example,the environment described above in reference to FIG. 5 may be at leastpartially implemented using device 1300. As another example, the devicesdescribed above in reference to FIG. 1 and FIG. 5 may be at leastpartially implemented using device 1300. As still another example, theprocesses described in reference to FIG. 7 , FIG. 8 , FIG. 9 , FIG. 10 ,FIG. 11 , and FIG. 12 may be at least partially implemented using device1300.

Device 1300 may be implemented using various appropriate elements and/orsub-devices. For instance, device 1300 may be implemented using one ormore personal computers (PCs), servers, mobile devices (e.g.,smartphones), tablet devices, wearable devices, and/or any otherappropriate devices. The various devices may work alone (e.g., device1300 may be implemented as a single smartphone) or in conjunction (e.g.,some components of the device 1300 may be provided by a mobile devicewhile other components are provided by a server).

As shown, device 1300 may include at least one communication bus 1310,one or more processors 1320, memory 1330, input components 1340, outputcomponents 1350, and one or more communication interfaces 1360.

Bus 1310 may include various communication pathways that allowcommunication among the components of device 1300. Processor 1320 mayinclude a processor, microprocessor, microcontroller, digital signalprocessor, logic circuitry, and/or other appropriate processingcomponents that may be able to interpret and execute instructions and/orotherwise manipulate data. Memory 1330 may include dynamic and/ornon-volatile memory structures and/or devices that may store data and/orinstructions for use by other components of device 1300. Such a memorydevice 1330 may include space within a single physical memory device orspread across multiple physical memory devices.

Input components 1340 may include elements that allow a user tocommunicate information to the computer system and/or manipulate variousoperations of the system. The input components may include keyboards,cursor control devices, audio input devices and/or video input devices,touchscreens, motion sensors, etc. Output components 1350 may includedisplays, touchscreens, audio elements such as speakers, indicators suchas light-emitting diodes (LEDs), printers, haptic or other sensoryelements, etc. Some or all of the input and/or output components may bewirelessly or optically connected to the device 1300.

Device 1300 may include one or more communication interfaces 1360 thatare able to connect to one or more networks 1370 or other communicationpathways. For example, device 1300 may be coupled to a web server on theInternet such that a web browser executing on device 1300 may interactwith the web server as a user interacts with an interface that operatesin the web browser. Device 1300 may be able to access one or more remotestorages 1380 and one or more external components 1390 through thecommunication interface 1360 and network 1370. The communicationinterface(s) 1360 may include one or more APIs that may allow the device1300 to access remote systems and/or storages and also may allow remotesystems and/or storages to access device 1300 (or elements thereof).

It should be recognized by one of ordinary skill in the art that any orall of the components of computer system 1300 may be used in conjunctionwith some embodiments. Moreover, one of ordinary skill in the art willappreciate that many other system configurations may also be used inconjunction with some embodiments or components of some embodiments.

In addition, while the examples shown may illustrate many individualmodules as separate elements, one of ordinary skill in the art wouldrecognize that these modules may be combined into a single functionalblock or element. One of ordinary skill in the art would also recognizethat a single module may be divided into multiple modules.

Device 1300 may perform various operations in response to processor 1320executing software instructions stored in a computer-readable medium,such as memory 1330. Such operations may include manipulations of theoutput components 1350 (e.g., display of information, haptic feedback,audio outputs, etc.), communication interface 1360 (e.g., establishing acommunication channel with another device or component, sending and/orreceiving sets of messages, etc.), and/or other components of device1300.

The software instructions may be read into memory 1330 from anothercomputer-readable medium or from another device. The softwareinstructions stored in memory 1330 may cause processor 1320 to performprocesses described herein. Alternatively, hardwired circuitry and/ordedicated components (e.g., logic circuitry, ASICs, FPGAs, etc.) may beused in place of or in combination with software instructions toimplement processes described herein. Thus, implementations describedherein are not limited to any specific combination of hardware circuitryand software.

The actual software code or specialized control hardware used toimplement an embodiment is not limiting of the embodiment. Thus, theoperation and behavior of the embodiment has been described withoutreference to the specific software code, it being understood thatsoftware and control hardware may be implemented based on thedescription herein.

While certain connections or devices are shown, in practice additional,fewer, or different connections or devices may be used. Furthermore,while various devices and networks are shown separately, in practice thefunctionality of multiple devices may be provided by a single device orthe functionality of one device may be provided by multiple devices. Inaddition, multiple instantiations of the illustrated networks may beincluded in a single network, or a particular network may includemultiple networks. While some devices are shown as communicating with anetwork, some such devices may be incorporated, in whole or in part, asa part of the network.

Some implementations are described herein in conjunction withthresholds. To the extent that the term “greater than” (or similarterms) is used herein to describe a relationship of a value to athreshold, it is to be understood that the term “greater than or equalto” (or similar terms) could be similarly contemplated, even if notexplicitly stated. Similarly, to the extent that the term “less than”(or similar terms) is used herein to describe a relationship of a valueto a threshold, it is to be understood that the term “less than or equalto” (or similar terms) could be similarly contemplated, even if notexplicitly stated. Further, the term “satisfying,” when used in relationto a threshold, may refer to “being greater than a threshold,” “beinggreater than or equal to a threshold,” “being less than a threshold,”“being less than or equal to a threshold,” or other similar terms,depending on the appropriate context.

No element, act, or instruction used in the present application shouldbe construed as critical or essential unless explicitly described assuch. An instance of the use of the term “and,” as used herein, does notnecessarily preclude the interpretation that the phrase “and/or” wasintended in that instance. Similarly, an instance of the use of the term“or,” as used herein, does not necessarily preclude the interpretationthat the phrase “and/or” was intended in that instance. Also, as usedherein, the article “a” is intended to include one or more items and maybe used interchangeably with the phrase “one or more.” Where only oneitem is intended, the terms “one,” “single,” “only,” or similar languageis used. Further, the phrase “based on” is intended to mean “based, atleast in part, on” unless explicitly stated otherwise.

The foregoing relates to illustrative details of exemplary embodimentsand modifications may be made without departing from the scope of thedisclosure. Even though particular combinations of features are recitedin the claims and/or disclosed in the specification, these combinationsare not intended to limit the possible implementations of thedisclosure. In fact, many of these features may be combined in ways notspecifically recited in the claims and/or disclosed in thespecification. For instance, although each dependent claim listed belowmay directly depend on only one other claim, the disclosure of thepossible implementations includes each dependent claim in combinationwith every other claim in the claim set.

I claim:
 1. A device, comprising: one or more processors configured to:receive a media item; extract media content from the media item; receivesource indication information associated with a publisher of the mediaitem; validate the received source indication information by:identifying a set of associated validation rules, the set of associatedvalidation rules including an identity validation rule; identifying aset of associated validation resources, the set of associated validationresources including an identity validation resource; extractingvalidation information based on the received source indicationinformation; applying the set of associated validation rules to validatethe source indication information by: sending a request to at least onevalidation resource from the set of associated validation resources,wherein the at least one validation resource comprises the identityvalidation resource; receiving a validation message from the at leastone validation resource; and applying at least one validation rule fromthe set of associated validation rules, the at least one validation rulecomprising the identity validation rule; generate augmented mediacontent that includes the source indication information; and generateand store an augmented media item that includes the augmented mediacontent and the extracted media content.
 2. The device of claim 1,wherein the media item comprises audio content, audiovisual content, orgraphic content.
 3. The device of claim 1, wherein the augmented mediacontent comprises audio content, audiovisual content, or graphiccontent.
 4. The device of claim 1, wherein the identity validationresource is a payment processing resource, a social media site, a searchengine, or an email account provider.
 5. The device of claim 4, whereingenerating augmented media content based on the source informationcomprises generating text-based validation information based on thereceived validation message.
 6. The device of claim 5, whereingenerating augmented media content based on the source informationfurther comprises converting the text-based validation information toaudio content.
 7. The device of claim 6, wherein the source informationcomprises at least one of a name, username, email address, or publishingplatform.
 8. A non-transitory computer-readable medium, storing aplurality of processor executable instructions to: receive a media item;extract media content from the media item; receive source indicationinformation associated with a publisher of the media item; validate thereceived source indication information by: identifying a set ofassociated validation rules, the set of associated validation rulesincluding an identity validation rule; identifying a set of associatedvalidation resources, the set of associated validation resourcesincluding an identity validation resource; extracting validationinformation based on the received source indication information;applying the set of associated validation rules to validate the sourceindication information by: sending a request to at least one validationresource from the set of associated validation resources, wherein the atleast one validation resource comprises the identity validationresource; receiving a validation message from the at least onevalidation resource; and applying at least one validation rule from theset of associated validation rules, the at least one validation rulecomprising the identity validation rule; generate augmented mediacontent that includes the source indication information; and generateand store an augmented media item that includes the augmented mediacontent and the extracted media content.
 9. The non-transitorycomputer-readable medium of claim 8, wherein the media item comprisesaudio content, audiovisual content, or graphic content.
 10. Thenon-transitory computer-readable medium of claim 8, wherein theaugmented media content comprises audio content, audiovisual content, orgraphic content.
 11. The non-transitory computer-readable medium ofclaim 8, wherein the validation rules comprise a first rule thatrequires user verification and a second rule that requires indication ofsourcing information.
 12. The non-transitory computer-readable medium ofclaim 11, wherein generating augmented media content based on the sourceinformation comprises generating text-based validation information basedon the received validation message.
 13. The non-transitorycomputer-readable medium of claim 12, wherein generating augmented mediacontent based on the source information further comprises converting thetext-based validation information to audio content.
 14. Thenon-transitory computer-readable medium of claim 13, wherein the sourceinformation comprises at least one of a name, username, email address,or publishing platform.
 15. A method comprising: receiving a media item;extracting media content from the media item; receiving sourceindication information associated with a publisher of the media item;validating the received source indication information by: identifying aset of associated validation rules, the set of associated validationrules including an identity validation rule; identifying a set ofassociated validation resources, the set of associated validationresources including an identity validation resource; extractingvalidation information based on the received source indicationinformation; applying the set of associated validation rules to validatethe source indication information by: sending a request to at least onevalidation resource from the set of associated validation resources,wherein the at least one validation resource comprises the identityvalidation resource; receiving a validation message from the at leastone validation resource; and applying at least one validation rule fromthe set of associated validation rules, the at least one validation rulecomprising the identity validation rule; generating augmented mediacontent that includes the source indication information; and generatingand storing an augmented media item that includes the augmented mediacontent and the extracted media content.
 16. The method of claim 15,wherein the media item comprises audio content, audiovisual content, orgraphic content.
 17. The method of claim 15, wherein the augmented mediacontent comprises audio content, audiovisual content, or graphiccontent.
 18. The method of claim 15, wherein: the validation rulescomprise a rule that requires indication of sourcing information theaugmented media content comprises audio content that is appended to themedia item, and the audio content comprises the required indication ofsourcing information.
 19. The method of claim 18, wherein generatingaugmented media content based on the source information comprisesgenerating text-based validation information based on the receivedvalidation message.
 20. The method of claim 15, wherein: generatingaugmented media content based on the source information furthercomprises converting the text-based validation information to audiocontent, and the source information comprises at least one of a name,username, email address, or publishing platform.